Target Neighbor Consistent Feature Weighting for Nearest Neighbor Classification

We consider feature selection and weighting for nearest neighbor classifiers. Atechnical challenge in this scenario is how to cope with discrete update of nearestneighbors when the feature space metric is changed during the learning process.This issue, called the target neighbor change, was not properly addressed in theexisting feature weighting and metric learning literature. In this paper, we proposea novel feature weighting algorithm that can exactly and efficiently keep track ofthe correct target neighbors via sequential quadratic programming. To the bestof our knowledge, this is the first algorithm that guarantees the consistency be-tween target neighbors and the feature space metric. We further show that theproposed algorithm can be naturally combined with regularization path tracking,allowing computationally efficient selection of the regularization parameter. Wedemonstrate the effectiveness of the proposed algorithm through experiments.